Towards an Automatic Sign Language Translation System

This paper outlines a concept of a continuous sign language translation system. The system consists of two main components. One of these is a video-based recognition tool, which aims for automatic signer dependent recognition of sign language sentences, based on a lexicon of 100 signs of German Sign Language. A single colour video camera is used for image recording. The recognition is based on Hidden Markov Models concentrating on manual sign parameters. The video-based recognition tool achieves an accuracy of 91.6%. The second main component is a translation tool, that translates from sequences of sign identiiers (provided by the recognition tool) into text. It uses two types of statistical information sources: a translation model and a language model. The translation model is decomposed into a lexical model and an alignment model. Experiments on tasks with comparable complexity yeald accuracies between 88 and 100%.